Skip to content

test(node): real-node deny harness for trust-boundary regressions (owner-gated mutations, path-scoped reads, client-surfaced denials)#194

Open
beardthelion wants to merge 43 commits into
mainfrom
feat/real-node-deny-harness
Open

test(node): real-node deny harness for trust-boundary regressions (owner-gated mutations, path-scoped reads, client-surfaced denials)#194
beardthelion wants to merge 43 commits into
mainfrom
feat/real-node-deny-harness

Conversation

@beardthelion

@beardthelion beardthelion commented Jul 12, 2026

Copy link
Copy Markdown
Collaborator

What

A real-socket, end-to-end security regression harness for gitlawb-node. It boots a real node on 127.0.0.1:0, drives trust-boundary DENY paths through a real reqwest client with real RFC-9421 signing, and asserts both the refusal status and that no withheld data leaks. It turns the per-PR real-node-verify step into executed tests, and covers ground tower::oneshot can't: the full production middleware stack over an actual socket, plus a systematic no-empty-200 (denial-as-success) assertion.

Why the crate split

gitlawb-node was binary-only, so an out-of-crate integration test could not reach build_router / AppState / Config. The first commit splits it into lib+bin: the module tree and boot logic move to src/lib.rs (exposing a minimal run() plus the boot surface), and src/main.rs becomes a thin #[tokio::main] shim. No behavior change, the 488-test node suite stays green and the production binary is unaffected. The harness itself lives behind a test-harness feature so its spawn surface never compiles into the release binary.

Coverage

Fourteen executed cases, one strong case per invariant plus a high-value owner-gate fan-out:

  • Denials are never rendered as empty/success: unsigned git-receive-pack rejected 401, and an anonymous /ipfs/{cid} read of a withheld blob denied 404 with no leak.
  • Mutations are owner-gated, not merely authenticated: a validly-signed non-owner is rejected 403 on set_visibility, plus protect/unprotect branch, webhook create/delete, and visibility removal. Each carries an owner-reachability check so a 403 from an earlier layer cannot masquerade as a pass.
  • Reads and replication gate on the requested path: a withheld blob read denied 404 with no leak, the same withhold over the content-addressed /ipfs surface, and the git-upload-pack replication path where the served pack must omit the withheld blob while keeping the sibling public one.

Every case is mutation-verified load-bearing: the specific gate was broken, the test observed to go red (the secret leaking, or the withheld object appearing in the pack), then reverted.

Notes for review

  • The upload-pack case drives the git-upload-pack POST directly (v0 stateless-RPC) instead of git clone, because a default git clone negotiates protocol v2 and hangs against the node's v0 server. That hang (rather than a clean error) may be worth a separate look if standard-client interop matters. The assertion is packfile-aware (git index-pack + verify-pack), not a raw byte scan, since a leaked OID would otherwise hide inside the zlib stream.
  • CI runs the harness explicitly with --features test-harness, since cargo test --workspace skips it by design.
  • Deliberately deferred: the lower-impact owner-gates and read surfaces (labels, PR comments, list_visibility, and similar) keep their existing source-level authz-table guard and gate-helper unit tests, where a full-stack case per endpoint adds near-zero marginal safety. Replica register/unregister were excluded because they are signer-self, not owner-gated.

Summary by CodeRabbit

  • Tests
    • Added a real-node deny-harness suite with end-to-end coverage for authorization denials, request signature validation, protected content access, and owner vs. non-owner mutation safeguards.
    • Added denial assertion helpers to verify correct 4xx responses and prevent leakage of withheld tokens/identifiers.
    • Added replication checks to ensure protected objects are excluded from returned packfiles.
  • CI / Release Engineering
    • Extended PR CI to run the deny-harness regression suite on every pull request using the existing Postgres test service.

t added 9 commits July 12, 2026 15:54
…can spawn a real node

Move the module tree and boot logic from main.rs into a new lib.rs crate
root exposing the boot surface (build_router, AppState, Config, migrations)
as pub; main.rs becomes a thin #[tokio::main] shim over run(). No behavior
change: both targets build and the full node suite (488 tests) stays green.

Prerequisite for the real-node deny harness (U1).
…-U4, U5a)

Add a feature-gated (test-harness) spawn surface (src/test_harness.rs) that
boots a real node on 127.0.0.1:0 over an ephemeral #[sqlx::test] pool through
the production axum::serve stack with connect-info, and an integration crate
(tests/deny_harness.rs) that drives deny paths with a real reqwest client:

- U2 signing client: wraps gitlawb_core::http_sig::sign_request for reqwest;
  self-checks that a valid signature clears require_signature and a tampered
  body is rejected (400 content_digest_mismatch).
- U3 spawn_node: real socket, p2p disabled, per-test DB, shutdown-on-drop.
- U4 assert_denied: 4xx AND body-no-leak AND not-empty-200 (INV-8); pure core
  unit-tested for clean-403 / empty-200 / leaking-403 / wrong-status.
- U5a INV-8: unsigned git-receive-pack is denied 401 with no leak.

Widens the three cfg(test) test builders (Db/RepoStore::for_testing,
run_migrations) to also compile under the feature. No production behavior
change: prod build (no feature) excludes test_harness; node suite stays green
(488) and the 7 integration tests pass.
A validly signed non-owner PUT /visibility is rejected 403 by require_owner
(no x-ucan, so require_ucan_chain passes through to the gate); the owner's
signed PUT reaches the handler (reachability proof, guards against a 404/415
masquerading as a pass). Adds seed_repo/withhold_path seeding helpers to the
test harness. Mutation-verified load-bearing: with require_owner forced Ok the
non-owner PUT returns 201 and the INV-8 assertion flips the test RED.
Adds seed_bare_repo (shells git to build a real bare repo at the served path,
sha1 or sha256 object format) and two INV-2 deny cases over the real stack:

- U7: a public repo with a /secret/** withhold rule denies an anonymous blob
  read of the withheld path (404) with no content/OID leak, while the sibling
  public path is served (path-scoped, not blanket).
- U5b: the same withhold denies an anonymous /ipfs/{cid} read of the withheld
  blob's content-addressed id (404, no leak), while the public blob's CID is
  served. Completes U5 (INV-8) alongside U5a.

Both mutation-verified load-bearing: forcing visibility_check to allow leaks
the secret at 200 and the INV-8 assertion flips each test RED.
Drives the git-upload-pack POST directly (v0 stateless-RPC: want HEAD, flush,
done) via a bounded reqwest client rather than a vanilla `git clone` (which
negotiates protocol v2 and deadlocks against the node's v0 server, and would
otherwise wedge the suite). The served pack is indexed with git index-pack and
its objects listed with verify-pack -v: a packfile-aware assertion, since a raw
byte scan cannot see an OID inside the zlib-compressed stream.

A public repo with a /secret/** withhold rule must serve a pack that omits the
withheld blob's object while keeping the sibling public blob. Mutation-verified
load-bearing: forcing visibility_check to allow puts the withheld blob back in
the pack and flips the test RED.

Completes the harness (8 units, 11 integration tests). Prod build (no feature)
and the 488-test node suite stay green.
cargo test --workspace skips the harness because it lives behind the
test-harness feature (kept off the production binary). Add an explicit step
that runs it with the feature and the same Postgres service, so the INV-1/
INV-2/INV-8 trust-boundary regression cases execute on every PR instead of
only when run by hand.
Add unit tests for the two remaining check_denied branches: a non-4xx expected
status is rejected as a test bug, and an empty withheld token is skipped rather
than matching every body. Closes the last unexecuted branches in the deny
assertion.
Fan-out of U6 to the security-sensitive owner-gated mutations that had only the
source-level authz-table guard and no runtime deny test: protect_branch,
unprotect_branch, create_webhook, delete_webhook, remove_visibility. Each
rejects a validly-signed non-owner with 403 and lets the owner reach the
handler (not 403). Mutation-verified load-bearing on their shared root gate:
did_matches forced true opens all five (non-owner protect_branch returns 201)
and the test flips RED.

Replica register/unregister were intentionally excluded: they are signer-self
(you register your own node), not owner-gated, so there is no owner-deny to
assert.
@coderabbitai

coderabbitai Bot commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

The node startup code moves into a reusable library, while a feature-gated TestNode enables real HTTP, database, and Git integration tests. New deny-harness cases cover signatures, authorization, withheld data, cloning, and owner gates, and CI runs the suite against Postgres.

Changes

Reusable node library and boot lifecycle

Layer / File(s) Summary
Reusable node library and boot lifecycle
crates/gitlawb-node/src/lib.rs, crates/gitlawb-node/src/main.rs
Node startup, shutdown, database retry, degraded serving, metrics, peer operations, operator setup, identity persistence, and related tests move into the library; the binary delegates to gitlawb_node::run().

Feature-gated harness runtime and CI wiring

Layer / File(s) Summary
Feature-gated harness runtime and CI wiring
crates/gitlawb-node/Cargo.toml, crates/gitlawb-node/src/db/mod.rs, crates/gitlawb-node/src/git/repo_store.rs, crates/gitlawb-node/src/test_harness.rs, .github/workflows/pr-checks.yml
The test-harness feature exposes test constructors and a TestNode that serves an ephemeral node, seeds database records and bare repositories, and runs in CI with the Postgres service.

Signed requests and denial-path integration tests

Layer / File(s) Summary
Signed requests and denial-path integration tests
crates/gitlawb-node/tests/deny_harness.rs, crates/gitlawb-node/tests/support/*
Shared RFC-9421 signing and denial assertions support end-to-end checks for signature failures, receive-pack authorization, withheld reads and clones, and owner-gated mutations.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant CI
  participant DenyHarness
  participant TestNode
  participant PostgreSQL
  CI->>DenyHarness: run deny_harness with test-harness
  DenyHarness->>TestNode: spawn_node(pool)
  TestNode->>PostgreSQL: run migrations and seed state
  DenyHarness->>TestNode: send signed or anonymous requests
  TestNode-->>DenyHarness: return denial or filtered Git response
Loading

Possibly related PRs

  • Gitlawb/node#57: Modifies the same CI test-job orchestration area.
  • Gitlawb/node#119: Covers the receive-pack, Git gating, and RFC-9421 signing behavior exercised by this harness.

Suggested labels: sev:medium, kind:security

Suggested reviewers: jatmn

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly summarizes the main change: a real-node deny harness for trust-boundary regressions.
Description check ✅ Passed The description is detailed and covers what changed, why, coverage, and reviewer notes, even though it doesn't mirror the template headings exactly.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/real-node-deny-harness

Comment @coderabbitai help to get the list of available commands.

@beardthelion beardthelion added crate:node gitlawb-node — the serving node and REST API kind:test Test coverage or harness labels Jul 12, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
crates/gitlawb-node/tests/deny_harness.rs (1)

36-37: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win

Inconsistent request timeouts across the suite.

Only the clone test (line 344-347) builds its reqwest::Client with an explicit timeout, with a comment explaining why (avoiding a wedged suite). Every other test here (e.g. this one, and lines 67, 98, 123, 192, 249, 420) uses reqwest::Client::new() with no timeout. If the real node under test ever hangs on any of these paths, the test blocks until the 45-minute CI job timeout instead of failing fast with a clear cause.

♻️ Suggested fix: a shared bounded client helper
fn bounded_client() -> reqwest::Client {
    reqwest::Client::builder()
        .timeout(std::time::Duration::from_secs(30))
        .build()
        .expect("client builds")
}

Then swap each reqwest::Client::new() in this file for bounded_client().

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/gitlawb-node/tests/deny_harness.rs` around lines 36 - 37, Introduce a
shared bounded client helper in the deny harness, such as bounded_client, that
builds reqwest::Client with a 30-second timeout. Replace every
reqwest::Client::new() usage in this file, including the clone test, with the
helper while preserving the existing request behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/gitlawb-node/src/lib.rs`:
- Around line 539-572: Update the HTTP shutdown flow around axum::serve and the
with_graceful_shutdown future to enforce the configured grace duration, aborting
the server drain when it expires instead of waiting indefinitely for long-lived
requests. Use the existing grace value derived from config.shutdown_grace_secs
and remove the unused grace discard while preserving normal shutdown signaling
and serve_result handling.
- Around line 1007-1027: Update the identity-key creation and loading flow
around Keypair generation and key_path.exists() to eliminate the TOCTOU race and
disclosure window: create the file with OpenOptions::create_new(true) and Unix
mode 0o600, write the PEM through that handle, and handle AlreadyExists by
retrying the existing-key load path. When loading an existing key, validate or
tighten its permissions to 0600 before reading it, while preserving the existing
PEM parsing and error behavior.

---

Nitpick comments:
In `@crates/gitlawb-node/tests/deny_harness.rs`:
- Around line 36-37: Introduce a shared bounded client helper in the deny
harness, such as bounded_client, that builds reqwest::Client with a 30-second
timeout. Replace every reqwest::Client::new() usage in this file, including the
clone test, with the helper while preserving the existing request behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: da2a7166-0514-4eaf-9449-ef5be4e258e0

📥 Commits

Reviewing files that changed from the base of the PR and between ad7c2b2 and 532627f.

📒 Files selected for processing (11)
  • .github/workflows/pr-checks.yml
  • crates/gitlawb-node/Cargo.toml
  • crates/gitlawb-node/src/db/mod.rs
  • crates/gitlawb-node/src/git/repo_store.rs
  • crates/gitlawb-node/src/lib.rs
  • crates/gitlawb-node/src/main.rs
  • crates/gitlawb-node/src/test_harness.rs
  • crates/gitlawb-node/tests/deny_harness.rs
  • crates/gitlawb-node/tests/support/assert.rs
  • crates/gitlawb-node/tests/support/mod.rs
  • crates/gitlawb-node/tests/support/signing.rs

Comment thread crates/gitlawb-node/src/lib.rs Outdated
Comment thread crates/gitlawb-node/src/lib.rs Outdated

@jatmn jatmn left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found issues that need to be addressed before this is ready.

Findings

  • [P1] Create identity keys atomically with owner-only permissions
    crates/gitlawb-node/src/lib.rs:1007
    The new library retains the existing exists()fs::write() flow. On Unix, fs::write creates the PEM using umask-derived permissions and only then changes it to 0600; a local process can read the node private key in that window. The separate existence check also lets concurrent node starts overwrite each other's generated identity. Create the file with create_new and mode 0600, then handle AlreadyExists by loading the winning key.

  • [P2] Enforce the configured HTTP shutdown grace period
    crates/gitlawb-node/src/lib.rs:539
    with_graceful_shutdown begins draining when the signal fires but has no deadline; the computed grace is explicitly discarded at line 571. A long-lived request can therefore prevent termination until the orchestrator hard-kills the process, defeating GITLAWB_SHUTDOWN_GRACE_SECS and risking interrupted cleanup. Bound the drain with that duration and force completion once it expires.

beardthelion pushed a commit that referenced this pull request Jul 13, 2026
- Create the node identity key atomically with create_new + mode 0600, closing
  the umask-derived 0644 disclosure window and the exists()->write overwrite
  race; on AlreadyExists load the winner's key (bounded retry so a loser can't
  read a half-written PEM) and tighten looser perms on load.
- Enforce the configured shutdown grace: bound the axum drain by grace measured
  from the signal (extracted as drive_serve_with_grace), abandoning in-flight
  requests once it expires instead of waiting indefinitely. Removes the
  discarded grace value.
- Route deny-harness reqwest clients through a shared bounded_client (30s
  timeout) so a wedged node path fails fast instead of hanging to the CI limit.

Tests: 0600-on-create, load-tighten, concurrent-start convergence (create race),
and grace-race abandon / normal-drain / signal-gated-clock. All RED-then-GREEN
by execution.
- Create the node identity key atomically with create_new + mode 0600, closing
  the umask-derived 0644 disclosure window and the exists()->write overwrite
  race; on AlreadyExists load the winner's key (bounded retry so a loser can't
  read a half-written PEM) and tighten looser perms on load.
- Enforce the configured shutdown grace: bound the axum drain by grace measured
  from the signal (extracted as drive_serve_with_grace), abandoning in-flight
  requests once it expires instead of waiting indefinitely. Removes the
  discarded grace value.
- Route the deny-harness reqwest clients through a shared bounded_client (30s
  timeout) so a wedged node path fails fast instead of hanging to the CI limit.

Tests: 0600-on-create, load-tighten, concurrent-start convergence (create race),
and grace-race abandon / normal-drain / signal-gated-clock. All RED-then-GREEN
by execution.
@beardthelion

Copy link
Copy Markdown
Collaborator Author

Addressed the review feedback in 30672cf.

@jatmn:

P1 (identity key). Created atomically with OpenOptions::create_new(true).mode(0o600) and written through that handle, so there is no umask-derived 0644 window. A lost race hits AlreadyExists and loads the winner's key instead of overwriting, with a short bounded retry so it can't read a half-written PEM; loading an existing key also tightens loose perms to 0600.

P2 (shutdown grace). The drain is now bounded by the configured grace, measured from the signal rather than server start (extracted as drive_serve_with_grace) and forced to complete once it expires. The discarded grace is gone and the misleading comment corrected.

Unit tests cover 0600-on-create, tighten-on-load, 8-thread concurrent-start convergence, and the grace abandon / normal-drain / signal-gated paths.

Also took CodeRabbit's nitpick: the deny-harness clients now go through a shared 30s bounded_client so a wedged route fails fast instead of hanging to the CI timeout.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/gitlawb-node/src/lib.rs`:
- Around line 1120-1125: Update the key-writing logic in create_new for both
Unix and non-Unix branches so any write_all failure removes the partially
written file at key_path before returning the error. Preserve the existing
contextual error and successful write behavior, and ensure cleanup is attempted
consistently in both branches.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 620c5ae9-78f1-476e-badb-3e054d1f583a

📥 Commits

Reviewing files that changed from the base of the PR and between 532627f and 30672cf.

📒 Files selected for processing (2)
  • crates/gitlawb-node/src/lib.rs
  • crates/gitlawb-node/tests/deny_harness.rs
🚧 Files skipped from review as they are similar to previous changes (1)
  • crates/gitlawb-node/tests/deny_harness.rs

Comment thread crates/gitlawb-node/src/lib.rs Outdated

@jatmn jatmn left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found issues that need to be addressed before this is ready.

Findings

  • [P1] Remove a failed first-write identity file
    crates/gitlawb-node/src/lib.rs:1123
    The new create_new(true) path fixes the original permission window, but if the PEM write itself fails after the file has been created, the just-created key path is left behind as an empty or partial PEM. Every later start then takes the key_path.exists() branch, retries parsing that same bad file in load_racing, and exits with invalid PEM key instead of generating a fresh identity. A transient ENOSPC/EIO/quota failure during first boot can therefore permanently wedge the node until an operator manually deletes the file. Please remove the newly-created file on write_all failure in both the Unix and non-Unix branches before returning the error.

  • [P2] Do not ignore failed key permission tightening
    crates/gitlawb-node/src/lib.rs:1059
    The load path now advertises that loose existing identity keys are tightened to 0600, but the set_permissions result is discarded. If the file is readable but chmod fails, for example on a read-only mount or an ownership/ACL mismatch, the node still reads and uses a world/group-readable private key while logging a normal "loaded existing identity" path. That leaves the exact key exposure this follow-up is trying to close. Please surface the chmod failure or otherwise verify the final mode before continuing with the key.

…r tightening (#194)

F1 (P1): create_new(true) closed the permission window, but a write_all failure
after the file was created left an empty/partial PEM behind. Every later start
then took the key_path.exists() branch, re-parsed that corrupt file in load_racing,
and exited 'invalid PEM key' instead of regenerating — a transient ENOSPC/EIO on
first boot permanently wedged the node. Extract write_key_or_cleanup, which removes
the just-created file on write failure, and wire it into both the unix and non-unix
create branches.

F2 (P2): the load path tightened a loose existing key to 0600 but discarded the
set_permissions result. A chmod that failed (read-only mount, ownership/ACL
mismatch) left a world/group-readable private key in use while logging a normal
'loaded existing identity'. Surface the tighten failure (propagate it) and add
ensure_key_mode_0600, which fails closed if the key is not 0600 after the attempt.

RED->GREEN: failed_write_removes_the_partial_key_file (a failed write leaves no
file; RED without the remove_file). loose_key_mode_is_rejected_not_used (a 0644 key
is rejected; RED without the mode check). Existing created_key_is_mode_0600,
existing_key_is_loaded_and_tightened, and concurrent_starts_converge_on_one_identity
stay green. Full node lib+bin suite 497 passed, fmt + clippy clean.
@beardthelion

Copy link
Copy Markdown
Collaborator Author

Both addressed on fb8685d.

Remove a failed first-write (F1). Extracted write_key_or_cleanup, which removes the just-created file when write_all fails, and wired it into both the Unix and non-Unix create branches. A transient ENOSPC/EIO on first boot now leaves no file, so the next start regenerates instead of re-parsing an empty/partial PEM forever. Covered by failed_write_removes_the_partial_key_file (an injected write error removes the file; RED with the remove_file disabled), and I confirmed the non-Unix branch type-checks by compiling it via a temporary cfg swap.

Do not ignore a failed permission tighten (F2). The load path now propagates the set_permissions error and then calls ensure_key_mode_0600, which fails closed if the key is not 0600 after the attempt — so a chmod that failed or silently no-opd (read-only mount, ACL mismatch) is refused rather than read and used exposed. Covered by loose_key_mode_is_rejected_not_used (a 0644 key is rejected, a 0600 key accepted; RED with the check disabled). existing_key_is_loaded_and_tightened still passes (a loose key on a writable mount is tightened and loaded).

One behavior note on F2: a loose key that genuinely cannot be tightened is now rejected rather than used, which narrows the original "never reject a loose key" leniency — but only in the exposed-and-unfixable case, which is the exposure this follow-up closes. The real ENOSPC/chmod-fail I/O triggers are not driven end to end (no portable fault injection), but the error handling is proven at the helper level and the wiring is a one-line pass-through of the write/chmod result into it.

RED->GREEN for each; no other production behavior changes.

@beardthelion
beardthelion requested a review from jatmn July 14, 2026 01:38

@jatmn jatmn left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found issues that need to be addressed before this is ready.

Findings

  • [P2] Do not fail concurrent startup after an arbitrary 100 ms key-write window
    crates/gitlawb-node/src/lib.rs:1123
    create_new exposes the final key path before the winner has completed write_all, and every other process that sees that inode gives up after 50 2-ms retries. On a slow or temporarily stalled filesystem, a winner can legitimately take longer than that interval, so all losing node starts return invalid PEM key even though the winning write later succeeds. This reintroduces an availability failure for the concurrent-start case the new code is intended to make safe. Keep retrying until a meaningful startup deadline, or publish a fully written temporary key atomically so readers never observe a partial final file.

The create path did create_new(final)+write_all, so the final path appeared as
an empty inode before the PEM was flushed, and a losing/fast-path start only
retried ~100ms (50x2ms). On a slow or stalled filesystem the winner's write can
exceed that window, so every other start failed boot with 'invalid PEM key'
(#194). Publish atomically instead: write the full PEM to a sibling temp, then
hard_link it into place. hard_link is atomic and fails if the target exists, so
the final path only ever appears COMPLETE (no partial-read window), a lost race
never clobbers the winner, and a crashed writer leaves only a temp rather than a
partial final that would wedge later starts. load_racing_key now polls on a
wall-clock KEY_RACE_DEADLINE (5s) instead of a fixed 100ms count. Hoisted
load_existing_key/load_racing_key to module level for testability.

Adversarial RED->GREEN: a 250ms-slow winner is waited out (RED at the old 100ms);
a reader watching the final never sees a partial file (RED with create_new+write);
a losing publish does not clobber the winner; 500 tests pass.
@beardthelion

Copy link
Copy Markdown
Collaborator Author

Confirmed and fixed at e3eac37.

The finding is right. The create path did create_new(final)+write_all, so the final path appears as an empty inode before the PEM is flushed, and load_racing only retried 50 × 2ms ≈ 100ms — a slow/stalled-FS winner can exceed that, and every other start then fails boot with invalid PEM key.

I took option (b), keeping the single-winner guarantee: write the full PEM to a sibling temp, then hard_link it into place. hard_link is atomic and fails if the target exists, which gives three things at once — the final path only ever appears complete (no partial-read window), a lost race never clobbers the winner, and a crashed writer leaves only the temp rather than a partial final that would wedge later boots. load_racing_key now polls on a wall-clock deadline (5s) instead of a fixed 100ms count. I hoisted load_existing_key/load_racing_key to module level so the deadline is directly testable.

Vetted both ways:

  • A 250ms-slow winner is now waited out — RED when I reverted the deadline to 100ms.
  • A reader watching the final path never observes an empty/partial file — RED when I reverted the publish to create_new(final)+write.
  • A losing publish does not clobber the winner (and temps are cleaned up); the existing convergence test stays green.

Full suite 500 pass; fmt and clippy -D warnings clean.

Two deliberate tradeoffs worth flagging:

  1. The atomic publish uses hard_link, so it now requires a filesystem that supports hard links. Every normal node key location (ext4/xfs/overlay/tmpfs) does; a hardlink-less mount would fail boot loudly with a clear error rather than silently. Happy to add a create_new+write fallback for that case if you'd prefer universal-FS support over failing loud.
  2. A genuinely corrupt existing key now blocks boot up to the 5s deadline before erroring (a valid key still parses on the first attempt, no delay).

@jatmn ready for another look.

@beardthelion
beardthelion requested a review from jatmn July 15, 2026 04:21

@jatmn jatmn left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found issues that need to be addressed before this is ready.

Findings

  • [P1] Recover from a stale pre-publication key file
    crates/gitlawb-node/src/lib.rs:1174
    The temporary path is deterministically derived from the PID and a process-local counter. If the process dies after writing .identity.pem.tmp.<pid>.0 but before the cleanup at line 1195, that file remains. A restarted container commonly runs as PID 1 again and starts its counter at zero, so create_new fails on the same stale name before the final key exists; the node cannot start until an operator deletes the temp file. Retry a new temporary name (or safely handle stale temps) so an interrupted first provision remains recoverable.

  • [P1] Make identity-key publication durable before exposing it
    crates/gitlawb-node/src/lib.rs:1185
    write_all followed by hard_link only atomically changes the namespace; neither the completed temporary file nor the containing directory is synced. A host/power crash after the link can therefore leave identity.pem durable while its PEM bytes are missing or truncated. On restart the existing-path branch retries that invalid key and then exits, recreating the permanent startup wedge this change is intended to eliminate. Sync the temporary file before linking and the parent directory after publication, with appropriate error/cleanup handling.

  • [P2] Wait for the spawned node before SQLx drops its database
    crates/gitlawb-node/src/test_harness.rs:48
    Dropping TestNode only sends the shutdown watch signal; the detached axum::serve task, which owns the router's PgPool, is never joined. #[sqlx::test] starts cleanup_test immediately after the async test future returns, so it can issue DROP DATABASE while that server still retains connections. SQLx prints and ignores that cleanup failure, making the new CI job leak test databases and race/flap under parallel execution. Retain the task handle and add an async shutdown/teardown path that signals, awaits the server, and releases the pool before the test returns.

…n TestNode teardown (#194)

- pick the temp key name with a bounded per-call retry instead of a
  process-global counter: a crashed start's leftover temp no longer wedges
  a PID-stable restart, and cross-container publishers stop colliding
- fsync the temp PEM before it can be linked and the key directory after
  publication, so a power crash cannot leave a durable-but-truncated
  identity.pem that permanently blocks startup
- TestNode::shutdown() joins the serve task and closes the pool before
  sqlx's DROP DATABASE cleanup runs; every deny-harness test now ends
  with it instead of relying on drop-only teardown

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
crates/gitlawb-node/src/test_harness.rs (1)

142-151: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win

Propagate axum::serve failures.

let _ = ...await hides I/O failures, making the task appear to exit successfully. The harness may report an unrelated connection failure—or pass teardown—without exposing that the server stopped unexpectedly. Return the result or expect it so shutdown surfaces server failures.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/gitlawb-node/src/test_harness.rs` around lines 142 - 151, Update the
spawned server task around axum::serve to propagate its awaited result instead
of discarding it with let _. Return the result or use expect so axum::serve I/O
failures remain visible to the test harness while preserving graceful shutdown
behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@crates/gitlawb-node/src/test_harness.rs`:
- Around line 142-151: Update the spawned server task around axum::serve to
propagate its awaited result instead of discarding it with let _. Return the
result or use expect so axum::serve I/O failures remain visible to the test
harness while preserving graceful shutdown behavior.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: cd43aeed-185d-43f3-af41-8a37b2da755f

📥 Commits

Reviewing files that changed from the base of the PR and between e3eac37 and 688b506.

📒 Files selected for processing (3)
  • crates/gitlawb-node/src/lib.rs
  • crates/gitlawb-node/src/test_harness.rs
  • crates/gitlawb-node/tests/deny_harness.rs
🚧 Files skipped from review as they are similar to previous changes (2)
  • crates/gitlawb-node/tests/deny_harness.rs
  • crates/gitlawb-node/src/lib.rs

t added 13 commits July 21, 2026 20:51
…d winners

A fallback winner stalled past the race deadline can have its round
claimed by a recovering peer: the peer renames the marker, quarantines
the in-progress final, and republishes. The winner's own-marker removal
now decides the round: removal succeeding linearizes it ahead of any
recovery, while any failure demotes it to Lost, after waiting out live
or stale .recovering claims so the follow-up load cannot observe the
pre-quarantine key. Demotion is always safe: absent interference the
Lost path re-loads the winner's own key.
A marker orphaned between its durability fsync and create_new(final),
or a claim left by a crashed recovery, must not linger to misclassify a
much-later corruption of a good key as an interrupted first write. Both
healthy-boot arms (successful load, and a Won publish on the generate
path) now sweep stale .publishing and .recovering entries, but only
after the sweeper itself has made the final durable: a loadable final
is not necessarily synced, and removing a live winner's marker before
its sync_all would recreate the marker-less partial-final wedge on
power loss. Any durability failure skips the sweep and leaves the
markers in place.
…recoverable

Three review findings against the marker protocol. Recovery now claims
every publishing marker and stale recovering claim (per-item bounded
claim names), so a live publisher's own-marker commit check can no
longer falsely succeed while its round is stolen through a stale
marker; a post-claim re-parse aborts to reload when a publisher
completed in the window. Claims count toward the crash signature and
are claimable by rename, so a recoverer crash between claim and
quarantine no longer recreates the permanent wedge. Claims are released
on every exit including publish failures, closing the leak that made
demoted publishers ride the full race deadline on a dead claim. The
crash signature also needs two consecutive observations, giving a live
mid-write publisher one poll of grace before a reader can classify its
round as a crash.
Add the missing coverage on recover_crashed_publish's degraded arms:
quarantine-name exhaustion must fall back to removing the bad final and
still regenerate, and a blocked removal must surface the chained error
loudly, with the leftover claim proven claimable on the next boot. Route
all marker-class name construction through publishing_prefix and
recovering_prefix as the single source of truth, state the dir-fsync
degradation caveat in publish_key_atomically's guarantee prose, and pin
the crash-signature short-circuit with a timing assertion so recovery
never silently regresses to riding the full race deadline.
…d keys

Second-model review findings. The healthy-boot sweep now age-gates
recovering claims (2x the race deadline; a live claim is always
younger), because sweeping a live recovery's claim cancelled a demoted
publisher's clearance wait and reopened two-sided divergence; publishing
markers keep the unconditional sweep since stealing one only forces a
safe demotion. The post-claim re-parse quarantines only on content-class
failures (a transient error there releases the claims and stays loud),
and a key completed inside the re-parse-to-quarantine window is detected
by parsing the quarantined bytes and restored via no-clobber link
instead of being discarded. The crash signature must now persist for
250ms of consecutive observations, above realistic mid-write latency on
shared volumes, so concurrent starts stop hijacking live publishes.
…p to owned inodes

The restore now republishes the quarantined bytes through the atomic
publisher, so every tier refuses an existing final and a concurrent
winner survives; a lost restore keeps the quarantine as forensics and
loads the winner. The fallback's post-write error arms remove the final
only after proving the name still refers to the inode this publish
created, so a failing publisher whose round was claimed away can no
longer unlink the recoverer's republished key. Pre-write arms keep
by-name removal: they complete inside the crash-signature persistence
window, before any recovery can have claimed the round.
…laim freshness

A failed write or fsync on the fallback path now removes nothing: the
partial final and its marker stay in place as exactly the crash state
boot recovery claims, quarantines, and regenerates from. That deletes
the stat-then-unlink cleanup whose race could destroy a recoverer's
republished key, and can no longer strand a marker-less partial final
when an unlink fails. Pre-write failures keep removal and disposal;
they complete inside the crash-signature persistence window. Claim
renames also stamp the claim's mtime to now, because rename preserves
the source mtime and an aged stale marker otherwise births a live claim
the healthy-boot sweep would treat as orphaned mid-round.
…ess stamp

Two PID-1 containers on a shared identity volume derive identical claim
names, and rename replaces an existing destination, so one recoverer's
release could unlink the other's live claim. Claim names now carry a
per-process nonce between pid and counter, so the release set can never
alias another process's claims. The claim-freshness stamp gains a write
fallback (a one-byte append bumps mtime where set_modified is
unsupported), and its failure warning names the real hazard. Also lock
the vanished-final re-parse arm with a dedicated regression test.
…tealing

A completed key stranded in quarantine by a failed or crash-interrupted
restore is no longer abandoned: the generate arm first scans quarantines
newest-first and republishes the first parseable one through the atomic
publisher, preserving identity continuity instead of silently minting a
fresh DID; unparseable quarantines stay as forensics. Recovering claims
become claimable only past the same age floor the sweep uses, so a
second starter can never steal a live recoverer's round mid-flight; a
claim-crash state recovers after the bounded age floor passes. The
crash-signature persistence doc now states the slow-mount trade-off
plainly: a pre-write window exceeding the floor yields churn that
converges, never divergence.
… waiter

Adoption of a stranded quarantined identity is now bounded by the claim
age floor, so a current round's completed key is preserved while a
historical forensic quarantine can never resurrect an old DID or defeat
the delete-identity-for-fresh-DID operator procedure. A healthy boot
also schedules one delayed re-sweep after the age floor passes, so
claims spared as possibly-live cannot linger as aged residue that would
let later real corruption of a good final masquerade as a crash state
and silently regenerate. The demotion waiter now shares the sweep's
liveness rule: it waits out claims younger than the age floor rather
than a flat race deadline, and skips aged orphans quickly.
…nes, fail closed on unsettled rounds

Every arm that returns a loaded, adopted, or reloaded key now funnels
through one sweep-then-return closure, so stale markers cannot outlive
any healthy boot path. Quarantines that provably lost their round (a
Lost restore, or siblings of an adopted quarantine) move to an inert
superseded class: bytes preserved as forensics, never adoptable, so the
delete-identity-for-fresh-DID operator procedure can no longer
resurrect an outcompeted key. The demotion waiter fails closed when its
bound expires while a live young claim persists: an unsettled round now
surfaces as a loud failed publish instead of resolving a key that disk
may be about to contradict.
… old forensics

Complete the supersede rule across its mirror arms: a lost adoption and
a restore-Won round now both demote every outcompeted adoptable
quarantine, so no arm leaves bytes the fresh-DID procedure could
resurrect. Live recovery rounds now heartbeat their claim mtimes every
quarter of the age floor under an RAII stop-and-join guard, making
age-as-orphan sound: an aged claim provably lost its recoverer, and a
stalled-but-live round on a slow volume can no longer be mistaken for
residue by the waiter, the sweep, or a competing recoverer. Superseded
forensic files are retention-swept after 24 hours, bounding growth on
long-lived key directories.
…s, key the waiter on absence

Both Won arms now supersede sibling quarantines before consuming the
chosen one, so a crash in the gap cannot leave an outcompeted key
adoptable. Plain quarantine forensics get the same 24h retention sweep
as superseded files, bounding growth on long-lived key directories. The
demotion waiter drops age logic entirely: it returns Ok only when every
claim file is gone and fails the publish closed while any claim
survives the bound, young or aged, so a heartbeat failure on a hostile
volume can no longer convert a live round into orphan residue mid-wait.
Doc prose reconciled with the shipped wait contract.
@beardthelion

Copy link
Copy Markdown
Collaborator Author

Fixed at 32903c4 (rebased onto the #195 test landing), 15 commits, each verified by execution at its head. Final-head validation: 581 lib tests plus 39 deny-harness integration tests pass with the test-harness feature, fmt and clippy clean. 62 new identity-key tests; every behavior-bearing one was observed failing before its fix, and the load-bearing guards are mutation-vetted (each goes red when its check is removed).

The finding is right, and it overruled round five's stated trade-off for the right reason. That trade-off rested on a partial first write being indistinguishable from a corrupted real key. The fix removes the indistinguishability instead of accepting the wedge: the fallback brackets its non-atomic window with a durable publish marker, created and fsynced before create_new(final). A boot that finds a marker beside a content-failed final provably has an interrupted first write: it claims the round, quarantines the final with bytes preserved, and regenerates. recovery_regenerates_after_crash_leaves_empty_final and its truncated-PEM sibling fail on the old code with the exact permanent wedge and pass now, in well under two seconds.

The must-not cases are locked by execution, not inspection:

  • A bad final with no marker keeps today's loud error, file untouched (bad_final_without_marker_still_fails_loudly).
  • A non-content load failure (transient EIO, EACCES, failed mode tighten) never recovers, marker or not; the class is a typed root error, not message matching (transient_error_with_marker_never_recovers).
  • A loadable final with a stale marker loads unchanged, and stale state is swept only after the sweeper itself fsyncs the final it loaded, because a page-cache-loadable final is not necessarily durable.

The concurrency half went through repeated adversarial review beyond the finding itself, and the protocol that survived is stricter than the first cut: recovery must atomically claim a round before touching anything, with per-process nonces in claim names so PID-1 containers on a shared volume cannot alias each other; live rounds heartbeat their claims under an RAII stop-and-join guard, which makes claim age a sound orphan signal for sweeping and claimability; the winner's own-marker removal is its commit check, and a demoted publisher waits for claim absence and fails closed if any claim survives the bound; post-write publish failures remove nothing and hand the marker-plus-partial state to boot recovery; a quarantined key that turns out complete is restored through the atomic no-clobber publisher; a stranded quarantine younger than the age floor is adopted for identity continuity, while anything that provably lost its round is superseded, so deleting identity.pem for a fresh DID can never resurrect an outcompeted key; forensic files are retention-swept after 24 hours.

Honest residuals, all stated in the code docs: on dir-fsync-hostile mounts the marker fsync is warn-and-continue, so recovery there narrows to SIGKILL-class crashes rather than never provisioning; a live publish whose pre-write window outlasts the 250ms crash-signature persistence can be hijacked by a concurrent starter, which converges through demotion and restore at the cost of churn; non-Unix keeps this file's existing reasoned-not-run posture. The adversarial passes had converged to P2-and-below for several rounds when I stopped them, but I would not claim the multi-actor lifecycle space is exhausted; extra scrutiny on the shared-volume interleavings is welcome.

One settled design call: the identity machinery is now large enough to deserve its own module, but I kept it in place this round so the diff stays reviewable against the finding it fixes; the split is queued as a follow-up.

t added 2 commits July 21, 2026 21:18
After merging main, the completeness fence flagged three repo-scoped list
GETs that #113 gated but the harness still treated as ungated public reads:
list_webhooks (authorize_repo_read + require_repo_owner), list_replicas and
list_protected_branches (authorize_repo_read). The runtime marker scan caught
its own hand-maintained PUBLIC_REPO_GETS list drifting stale.

Drive each real deny instead of exempting it:

- list_replicas and list_protected_branches take Option auth, so they slot in
  as standard ReadGate rows (anon and signed-non-reader both 404, owner 2xx,
  no-leak body).
- list_webhooks is doubly gated and 401s a headerless caller before any
  lookup, so a vanilla ReadGate row's hardcoded anon->404 leg does not fit.
  Add a per-Row anon_read signal (Deny404 default, Deny401 for auth-required
  reads) so its anon leg expects 401 while the signed-non-reader still drives
  the existence-hiding 404. Its owner layer is a sibling OwnerGate row on the
  public repo (signed stranger -> 403). The two share the /hooks path, so the
  consistency dedup key moves from (method, path) to (method, path, gate),
  which still catches an accidental same-gate double-registration.
- Empty PUBLIC_REPO_GETS: the three are now driven rows, classified via the
  driven-path set rather than declared public. The mechanism stays for the
  next genuinely-public repo-GET.

Each new row proven load-bearing: reverting the production gate turns the
matching hostile probe red (403->200 owner, 404->403/200 read), and flipping
list_webhooks' anon_read to Deny404 reddens its anon probe (got 401). Full
deny_harness green (39/39). No production changes.

@jatmn jatmn left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found issues that need to be addressed before this is ready.

Findings

  • [P2] Fail when the denial body cannot be read
    crates/gitlawb-node/tests/support/assert.rs:55
    assert_denied turns a resp.text() error into an empty body. A denial can send its expected 4xx headers and then reset or truncate the response body; this helper then accepts the status and observes no withheld token, so the new no-leak regression passes without inspecting the body that might contain the leak. Propagate the body-read failure before applying check_denied.

  • [P2] Close every ordinary spawned node before SQLx cleanup
    crates/gitlawb-node/tests/deny_harness.rs:154
    The protected-branch test returns without node.shutdown().await; the same is true for the PR-diff and registry-sweep tests. They therefore use TestNode::Drop, which cannot await/reap the Axum task or close its pool, while #[sqlx::test] immediately drops the test database. Server-held pool connections can race that cleanup and leak databases or make the new CI job flaky. End each normal test with the awaited shutdown (leaving only the tests intentionally exercising Drop as exceptions).

  • [P2] Bound the fixture setup requests as well as the probes
    crates/gitlawb-node/tests/support/probe.rs:319
    The registry test builds its fixture before it reaches its bounded probe client, but the fixture seeders still use unbounded reqwest::Client::new() calls. A hung create/claim/push route during setup can therefore wait until the 45-minute CI job timeout, despite the harness's stated fast-fail timeout contract. Reuse support::bounded_client() (or pass a bounded client through the seed helpers) for these setup requests.

t added 2 commits July 22, 2026 00:25
Resolve three review findings on the real-node deny harness, all test-only.

- assert_denied no longer folds a body-read error into an empty body: an
  unreadable denial (a mid-body reset/truncation) now fails loud instead of
  being certified leak-free by a withheld-token scan over "".

- The three sqlx::test cases that returned via TestNode's async Drop
  (signed_stranger_protected_branch_push_is_forbidden,
  get_pr_diff_withheld_path_is_denied,
  deny_bearing_registry_denies_hostile_and_admits_authorized) now end with
  node.shutdown().await, releasing the serve task's pool clones before sqlx's
  synchronous DROP DATABASE cleanup instead of racing it.

- A source-scrape guard, every_spawn_node_test_shuts_the_node_down, makes that
  contract load-bearing for every spawn_node test, with a closed allowlist for
  the two Drop-regression tests that deliberately skip shutdown. Removing any
  one shutdown() turns it RED (verified).

- The eight fixture seeders in support/probe.rs use the bounded client, so a
  hung setup route fails fast rather than running to the CI job timeout.

Full deny_harness suite green (40 tests).
An adversarial review flagged that the new every_spawn_node_test_shuts_the_
node_down guard could itself pass vacuously - the exact class it exists to
prevent (INV-21 on the guard).

- A spawn_node test whose only `node.shutdown(` occurrence was a comment
  (`// node.shutdown().await`) satisfied the raw substring check while still
  returning via Drop.
- An unbalanced `{` inside a string literal in a test body could over-extend
  the brace-matched body slice and swallow a later test's shutdown, clearing
  the earlier one (latent: no current test triggers it).

Both are closed by sanitizing the source through a new `code_only` pass
(blanks comment and string/raw-string content, length-preserving) before the
scan sees it, and the offender check is extracted to a pure
spawn_node_tests_missing_shutdown so it can be driven with synthetic sources.
Three adversarial unit tests pin both holes shut; stubbing code_only to a
no-op turns the comment and brace tests RED (verified), so the sanitizer is
load-bearing.

deny_harness suite green (43 tests); clippy clean.
@beardthelion

Copy link
Copy Markdown
Collaborator Author

All three are addressed in 8d81cbe, with a follow-up in 996e1b3. Full deny_harness suite is green (43 tests).

Fail when the denial body cannot be read (support/assert.rs): assert_denied now propagates the body-read failure instead of folding it into an empty string, so an unreadable denial can no longer pass the withheld-token check vacuously. A legitimately empty body is still Ok(""), so the empty-body 401 denials pass unchanged.

Close every ordinary spawned node before SQLx cleanup (deny_harness.rs): added node.shutdown().await to the three tests you named (protected-branch push, PR-diff, registry sweep). To keep the contract from drifting again, I added a completeness guard that scrapes this file and fails if any spawn_node test omits shutdown(), with a closed allowlist for the two Drop-regression tests that deliberately exercise Drop. It is load-bearing: removing any one shutdown() turns it red (verified), and the guard is itself unit-tested against a commented-out call and a brace-in-string body so it cannot pass vacuously.

Bound the fixture setup requests (support/probe.rs): the eight seeder clients now use the bounded support::bounded_client() (30s) instead of reqwest::Client::new(), so a wedged setup route fails fast instead of running to the CI job timeout.

@jatmn ready for another look.

@beardthelion
beardthelion requested a review from jatmn July 22, 2026 10:56

@jatmn jatmn left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found an issue that needs to be addressed before this is ready.

Findings

  • [P1] Reintegrate the harness branch with current main without restoring removed behavior
    .github/workflows/pr-checks.yml:3, .github/workflows/release.yml:49, crates/gl/src/cert.rs:151, crates/gl/src/init.rs:41, crates/gl/src/status.rs:147
    The PR head does not descend from the current PR base (its merge-base is an older commit), and the resulting base-to-head diff rolls back several safeguards that are already on main: merge-queue validation, certificate signature verification, OIDC npm publication, native/smoke-tested arm64 releases, and the branch/remote-aware gl behavior. These changes are outside the deny-harness scope and would reintroduce the failures those merged fixes addressed. Please rebase or otherwise reintegrate with current main, resolving the integration so these existing protections remain, then request a fresh review of the resulting full diff.

@beardthelion beardthelion changed the title test(node): real-node deny harness for trust-boundary regressions (INV-1/2/8) test(node): real-node deny harness for trust-boundary regressions (owner-gated mutations, path-scoped reads, client-surfaced denials) Jul 22, 2026
@beardthelion

Copy link
Copy Markdown
Collaborator Author

Reintegrated with current main at 8f7af96 (a merge of b484a24, keeping the branch's reviewed history rather than rewriting it, which I read as within your "rebase or otherwise reintegrate").

On the rollback concern: I believe that reading came from a two-dot comparison against the new main. The branch's merge-base diff never touched four of the five files you named. Post-merge, git diff origin/main -- .github/workflows/pr-checks.yml .github/workflows/release.yml crates/gl/src/cert.rs crates/gl/src/init.rs crates/gl/src/status.rs shows exactly one hunk, the added deny-harness CI step in pr-checks.yml, sitting alongside the merge-queue guards from main. Merge-queue validation, cert signature verification, OIDC npm publish, native arm64 releases, and the gl status/init/doctor fixes are all intact at main's versions.

Verified on the merged state: full workspace suites green (601 node tests among them), and the deny harness runs 43/43, matching the pre-merge count, with the shutdown-guard vacuity tests present by name in the output. One deliberate non-change: Cargo.lock still carries the 0.5.1 stamps because the manifest/lock version drift exists on main itself and #243 owns that sync; absorbing it here would have added an unreviewed hunk outside this PR's scope.

Ready for a fresh look at the full diff.

@beardthelion
beardthelion requested a review from jatmn July 22, 2026 20:54

@jatmn jatmn left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found issues that need to be addressed before this is ready.

Findings

  • [P1] Preserve the released 0.7.0 version baseline
    .release-please-manifest.json:2
    The current base is already tagged v0.7.0, but this head changes the release manifest and every shipped crate back to 0.6.0 and deletes the 0.7.0 changelog section. As a result, a release from this branch reports an already superseded version and release-please calculates from a stale baseline, which can collide with existing tags/packages. Rebase or regenerate the release metadata so it retains the base's 0.7.0 state.

  • [P1] Do not remove the gitlawb-core dependency-purity gate
    .github/workflows/pr-checks.yml:257
    This PR deletes core-deps-purity together with its checker and exhaustive allowlist, with no replacement. That job is the only check that rejects new normal dependencies in gitlawb-core, which is embedded by the node, gl, and the remote helper; normal test, clippy, and audit runs do not enforce this contract. A subsequent dependency addition will therefore pass PR CI and silently propagate to every shipped consumer.

  • [P2] Refuse symlinked identity-key paths before tightening permissions
    crates/gitlawb-node/src/lib.rs:1257
    metadata and set_permissions both follow GITLAWB_KEY symlinks. If an attacker can replace the configured key path in a writable directory, a privileged node start chmods the symlink target to 0600 before validating or parsing it. That permits filesystem-integrity denial of service against any service-readable target and is newly introduced by the key-hardening path. Open the key without following symlinks and verify it is a regular file before changing permissions or reading it.

  • [P2] Close the temp key before trying to remove it after a failed write
    crates/gitlawb-node/src/lib.rs:2031
    The error path calls write_key_or_cleanup while f is still open; that helper immediately calls remove_file. On Windows an open file cannot be removed, so a write or fsync failure leaves .{stem}.tmp.<pid>.<n} behind. Repeated restarts with a reused service/container PID can exhaust all 64 deterministic names and prevent recovery after storage becomes healthy. Drop the handle before cleanup (and keep the cleanup result observable) so the documented failed-write recovery works cross-platform.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

crate:node gitlawb-node — the serving node and REST API kind:test Test coverage or harness

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants